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Enhancing  Maritime  Domain  Awareness 

Abstract 

A  complex  mosaic  of  forces  will  affect  maritime  security  through  2020.  These 
include  economic  forces  such  as  (z)  illegal  international  migration-fueled  by 
tremendous  population  increases  in  developing  countries,  (if)  drug  smuggling,  and 
(Hi)  weapons  proliferation,  as  well  as  (z'v)  military  threats  posed  by  hostile  states, 
failed  states,  and  transnational  organizations  engaged  in  intelligence  gathering  or 
terrorist  activities.  The  President’s  National  Security  Strategy  indicates  that 
diverse  threats  such  as  these  must  be  countered  through  an  integrated  approach: 
Maritime  Domain  Awareness  (MDA).  MDA  will  engage  and  shape  this  dynamic 
and  expansive  environment  by  detecting  anomalies  and  deviations  from 
established  trends  and  patterns  in  commercial  and  military  traffic,  and  enabling 
commanders  to  take  appropriate  action  before  security  is  compromised  or  crises 
erupt.  In  line  with  the  theme  for  this  year’s  symposium,  C2  for  complex 
endeavors,  MDA  encompasses  a  diverse  set  of  organizations,  complex  processes, 
and  a  variety  of  analytic  and  collaborative  tools.  This  research  reported  here  will 
advance  organizational  structures,  procedures  and  C2  technologies  to  enhance 
global  MDA.  Our  approach  is  to  document  current  MDA  capabilities,  compare 
them  with  the  next  phase  of  expected  capabilities  based  on  integrating  new 
technologies,  and  document  the  gains. 

INTRODUCTION 

A  confluence  of  forces  will  affect  maritime  security  through  2020.  These  include  economic 
forces  such  as  (z)  illegal  international  migration-fueled  by  tremendous  population  increases  in 
developing  countries,  (z'z)  drug  smuggling,  and  (z'z'z)  weapons  proliferation  (Stubbs  &  Truver, 
1999),  as  well  as  (z'v)  military  threats  posed  by  hostile  states,  failed  states,  and  transnational 
organizations  engaged  in  intelligence  gathering  or  terrorist  activities  (White  House,  1998).  The 
President’s  National  Security  Strategy  indicates  that  diverse  threats  such  as  these  must  be 
countered  through  an  integrated  approach:  Maritime  Domain  Awareness  (MDA).  MDA  will 
engage  and  shape  this  dynamic  and  expansive  environment  by  detecting  anomalies  and 
deviations  from  established  trends  and  patterns  in  commercial  and  military  traffic,  and  enabling 
commanders  to  take  appropriate  action  before  security  is  compromised  or  crises  erupt  (DoN, 
2007).  In  line  with  the  theme  for  this  year’s  symposium,  C2  for  complex  endeavors,  MDA 
encompasses  a  diverse  set  of  organizations,  complex  processes,  and  a  variety  of  analytic  and 
collaborative  tools.  This  research  reported  here  will  advance  organizational  structures,  proce¬ 
dures  and  C2  technologies  to  enhance  global  MDA.  Our  approach  is  to  document  current  MDA 
capabilities,  compare  them  with  the  next  phase  of  expected  capabilities  based  on  integrating  new 
technologies,  and  document  the  gains. 

The  MDA  Prototype  Acceleration  Project  is  focused  on  initial  capability  fielding  of  prototype 
tools  developed  to  assist  with  various  aspects  of  MDA.  After  the  initial  assessment,  conducted  in 
2008,  selected  tools  will  subsequently  transition  to  current  and/or  new  Programs  of  Record  in 
FY10  and  beyond.  Two  fielding  spirals,  Spiral  1  and  Spiral  2,  were  designated  for  the 
acceleration  effort,  with  Spiral  1  commencing  with  the  publication  of  the  SECNAV 
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memorandum  on  17  May  2007.  The  Spiral  1  effort  focuses  on  establishing  a  set  of  core  net- 
centric  capabilities  that,  at  multiple  levels  of  security,  will  feed  and  fuse  many  data  streams  into  a 
coherent  common  operational  picture  for  Spiral  1  assessment  while  providing  improved 
analytical  and  collaboration  tools. 

The  purpose  of  this  research  is  to  develop  a  comprehensive  assessment  plan  and  conduct  an 
assessment  of  the  MDA  Spiral  1  technologies  and  associated  capabilities  to  ascertain  the  value 
added  by  the  various  tools.  Our  approach  is  to  conduct  the  assessment  at  the  system  of  systems 
(SoS)  level,  in  keeping  with  performance  and  effectiveness  areas  identified  in  the  MDA 
Prototype  Capability  Acquisition  Strategy  (vl.l)  of  17  August  2007.  This  entails  working  with 
the  MDA  Spiral  1  planners,  functional  area  managers,  site  managers,  labs,  and  test  organizations 
involved  with  developing,  validating,  and  assessing  definitions,  requirements,  and  designs  and 
reporting  test  and  evaluation  activities  for  MDA  FY08  Spiral  1. 

Background 

MDA  was  initiated  by  a  Presidential  Initiative  (White  House,  1998)  which  stimulated  a 
requirement  for  a  Quick  Reaction  Assessment  (QRA)  to  be  conducted  for  MDA  Spiral  1 
Acceleration  leave  behind  capabilities.  A  Secretary  of  the  Navy  Project  was  promulgated  and  is 
managed  by  Program  Executive  Office  (PEO),  command,  control,  computers,  communications, 
and  intelligence  (C4I)  to  increase  the  United  States  maritime  security.  MDA  will  develop  an 
enhanced  capability  to  identify  threats  in  the  Maritime  Domain  as  early  and  as  distant  from  our 
shores  as  possible  by  integrating  intelligence,  surveillance,  observation,  and  navigation  systems 
into  a  common  operating  picture  (COP)  accessible  throughout  the  U.S.  Government.  The  MDA 
Science  and  Technology  (S&T)  Prototype  effort  is  in  the  formative  stages  such  that  the  prototype 
capabilities  have  broadly  stated  capability  thresholds  and  objectives. 

Various  assessment  events  will  be  conducted  to  evaluate  the  existing  system  of  systems  ability  to 
meet  MDA  requirements  (with  selected  technology  insertions,  as  they  are  available  at  different 
venues  of  opportunity).  These  FY08  scheduled  events  will  provide  opportunities  to  establish  how 
well  the  MDA  tools  meet  the  MDA  stated  capabilities,  measures  of  effectiveness,  and  measures 
of  perfonnance.  Various  venues  of  opportunity  offer  events  that  permit  collecting,  collating, 
analyzing,  and  documenting  performance  and  effectiveness  data  related  to  the  MDA  tasks  to 
establish  a  baseline  of  effectiveness  and  performance  parameters.  This  baseline  data  will  provide 
opportunities  to  establish  more  specific  and  measurable  effectiveness  and  performance 
parameters  for  future  MDA  efforts  in  the  stated  focus  areas  of  monitoring,  collecting,  fusing, 
analyzing,  and  disseminating  MDA  data  to  the  decision  makers. 

MDA  Objectives 

The  campaign  plan  lists  the  following  National  and  Navy  MDA  objectives  and  critical  tasks.  At 
the  national  level,  MDA  objectives  include  the  ability  to  persistently  monitor  in  the  global 
maritime  domain,  vessels  and  craft,  cargo,  vessel  crews,  and  passengers.  This  monitoring  capa¬ 
bility  applies  to  all  identified  area  maritime  situation  awareness  areas  of  interest,  and  includes 
gaining  access  to  and  maintaining  data  on  vessels,  facilities,  and  infrastructure.  A  critical 
capability  entails  the  ability  to  collect,  fuse,  analyze,  and  disseminate  information  to  decision 
makers  to  facilitate  effective  understanding  among  all  nodes  in  the  organization,  and  to  access, 
develop  and  maintain  data  on  MDA-related  mission  performance. 
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Navy  MDA  Objectives  include  the  ability  to: 

•  Create  a  global  network  of  unclassified  regionally-based  maritime  infonnation  exchange 
partnerships  (with  framework)  which  net-centrically  integrates  regional  maritime 
situation  awareness  into  global  maritime  situation  awareness. 

•  Establish  an  institution  of  worldwide  standards  and  cooperation/coordination  for 
broadcast  of  vessel  position  and  identification. 

•  Improve  maritime  change  detection,  the  capability  to  discern  patterns,  changes,  and 
potential  threats. 

•  Improve  threat  awareness  (TA)  and  alert  maritime  partners  of  suspicious  behavior  and 
potential  threats. 

•  Secure  the  United  States  from  direct  attack  by  confronting  early  and  at  safe  distances, 
those  who  threaten  us. 

•  Secure  strategic  access  and  retain  global  freedom  of  action  by  ensuring  that  key  regions, 
lines  of  communication  and  the  global  commons  remain  accessible  to  all. 

•  Strengthen  existing  and  emerging  alliances  and  partnerships  to  address  common 
challenges. 

•  Establish  favorable  security  conditions  by  countering  aggression  or  coercion  (including 
corruption)  targeted  at  our  partners,  interests,  and  operating  forces  -  including  coalition. 

METHOD 

Our  approach  was  to  document  the  current  MDA  process,  as  it  is  currently  conducted,  so  that  we 
can  measure  the  operational  impact  of  new  Spiral  1  MDA  technologies.  Our  objective  is  to 
define  and  assess  future  MDA  processes  and  technologies  and  then  develop  a  refined  version  of 
the  process.  We  began  by  viewing  the  overall  MDA  process  in  the  context  of  a  tracking  scenario 
and  a  boarding  scenario  to  document  the  current,  present  state  of  MDA  processes  and 
capabilities.  The  ‘to  be’  (expected)  version  of  MDA  workflow  for  the  future  will  be  used  to 
develop  techniques,  tactics,  and  procedures  (TTPs)  and  to  make  recommendations  for  other 
related  changes,  e.g.,  training,  staffing,  etc.  These  documents  were  validated  in  a  process 
engineering  workshop  held  in  January  2008  at  the  Naval  Postgraduate  School,  Monterey,  CA, 
and  a  February  2008  workshop  to  align  the  model  with  the  current  process  model  for  Maritime 
Headquarters  with  Maritime  Operations  Centers  (MHQ  w/  MOC).  The  workflow  was  also 
translated  into  a  DoD  Architectural  Framework  (DoDAF)  process  database  (OV-6c).  The 
workflow  product  will  inform  experiments  and  fielding  of  an  initial  operating  capability. 
Specifically,  a  set  of  potential  stress  points  within  the  MDA  workflow  have  been  identified. 
These  will  be  a  focus  of  upcoming  experiments,  and  process  recommendations  concerning  these 
stress  points  have  been  drafted. 

Developing  Workflow  Process  Models 

Initial  visits  were  made  by  NPS  project  team  members  to  Commander,  Third  Fleet  (C3F), 
Commander,  Seventh  Fleet  (C7F),  U.S.  Naval  Force,  Central  Command  (NAVCENT),  and 
Office  of  Naval  Intelligence  (ONI).  Interviews  were  conducted  with  MDA  subject  matter  experts 
(SMEs)  and  MDA  workflow  process  models  were  developed.  SMEs  were  asked  to  describe  in  as 
much  detail  as  possible  how  MDA  is  accomplished  and  a  semi-structured  set  of  interview 
questions  was  used  to  obtain  detailed  information  about:  (1)  MDA  tasks,  (2)  the  entities  that 
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execute  them,  (3)  the  precedence  relationships  (or  flow)  between  those  tasks,  and,  when  possible, 
(4)  the  media  used  to  communicate  between  tasks,  (5)  the  products  of  those  tasks,  and  (6)  the 
potential  application  points  for  Spiral  1  technologies.  Table  1  lists  the  questions  posed  to  SMEs 
regarding  the  current  ‘as  is’  MDA  process  workflow. 

Table  1.  Questions  Posed  to  Subject  Matter  Experts  Regarding  the  Current 

‘As  Is’  MDA  Process  Workflow. 


1 .  What  information  do  you  request  from  external  sources? _ 

2.  What  information  do  you  produce  internally  at  this  point? _ 

3.  What  information  do  you  pass  to  external  entities  at  this  point? _ 

4.  Which  information  sources  (people,  technologies)  have  the  greatest  value  or  priority? 

5.  How  do  these  steps  fail,  when  they  fail  (e.g.,  timeliness,  completeness,  accuracy,  confliction  )? 

6.  What  is  the  frequency  of  each  task? _ 

7.  What  is  the  duration  of  each  task? _ 

8.  How  many  full-time  staff  are  required  by  each  task? _ 

9.  Which  Spiral  1  technologies  would  improve  your  operations  most?  How? 


Our  general  framework  for  developing  an  understanding  of  the  overall  MDA  process  is 
illustrated  in  Figure  1.  This  comprehensive  approach  considers  the  constraints  that  will  affect 
personnel  while  perfonning  the  tasks  in  addition  to  the  data /  infonnation  inputs  and  outputs  for 
each  node  in  the  organization  as  well  as  a  detailed  description  of  the  required  resources. 


Constraints 


Inputs 


*• 


Task 


A 


*•  Outputs 


Required  Resources 


Figure  1.  General  Framework  Used  for  MDA  Task  Workflow  Definition. 

The  NPS  research  team  elicited  additional  information  about  workflow  and  technology-related 
issues  in  interviews  with  Commander,  Pacific  Fleet  (COMPACFLT),  Maritime  Intelligence 
Fusion  Center,  Pacific  (MIFCPAC),  ONI,  and  NAVCENT.  Two  types  of  analyses  were 
conducted  using  interview  data.  A  qualitative  analysis  was  performed  to  identify  areas  of 
concern  and  generate  recommendations.  A  process  analysis  was  performed  to  define  the  current 
-  or  ‘as  is’  -  workflow.  The  objective  was  to  define  the  current  -  or  ‘as  is’  -  flow  of  MDA  tasks 
in  a  representative  sample  of  organizations,  and  to  capture  issues  of  concern  to  those 
organizations  regarding  MDA  Spiral  1  technologies.  Representatives  of  more  than  20 
organizations  reviewed  and  revised  the  workflow  products  in  two  workshops. 
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Figures  2-4  present  the  current  version  of  the  ‘as  is’  MDA  workflow  process  model.  Using  the 
observe,  orient,  decide,  act  (OODA)-loop  model  as  a  framework  for  presenting  the  MDA 
process,  Figure  2  shows  the  MDA  process  begins  with  generation  of  ‘tipper’  infonnation,  which 
can  occur  at  several  locations,  or  commands.  The  ‘decide’  phase  of  the  overall  process  begins 
when  the  Maritime  Operations  Center  (MOC)  Director,  directs  either  current  operations  (COPs) 
or  future  operations  (FOPs)  to  process  the  vessel  of  interest  (VOI).  This  process  includes  several 
requests  for  information  (RFI),  as  well  as  several  other  steps:  defining  the  course  of  action 
(COA),  and  handoff  of  the  mission  to  another  MOC.  Finally,  the  ‘act’  portion  of  the  process, 
involves  conducting  a  visit,  board,  search,  and  seizure  (VBSS)  on  the  VOI.  Each  step  depicted  in 
the  boxes  included  in  Figures  2-4  has  a  code  next  to  the  box  for  the  step  to  indicate  the  method  of 
communicating  the  infonnation:  briefs,  chat,  email,  face-to-face,  message  traffic,  phone,  or  other 
(to  be  detennined). 

Figure  3,  depicts  the  boarding  process  as  it  is  currently  done  (‘as-is’).  The  commands  that  have  a 
role  are  listed  along  the  left  axis  and  the  steps  involved  are  depicted  in  the  corresponding  rows 
for  each  command,  with  arrows  indicating  where  the  outcome  of  a  process  connects  to  the  next 
step,  and  the  method  used  to  communicate  infonnation.  Spiral  1  capabilities  and  technologies 
that  are  expected  to  be  added  to  enhance  the  various  steps  in  the  MDA  process  are  also  listed  in 
each  box  in  the  model.  These  were  determined  during  interviews  with  SMEs  at  the  commands 
included  in  the  MDA  process.  Figure  4  continues  the  depiction  of  the  MDA  workflow  process 
model  by  showing  the  steps  entailed  in  a  handoff  of  a  VOI  from  one  MOC  to  another  MOC. 

The  workflow  model  that  was  developed  focuses  mainly  on  NAVCENT  and  represents  the 
relatively  efficient  processes  used  by  NAVCENT.  (The  decision  to  focus  on  NAVCENT  was 
made  following  site  visits  to  C3F,  C7F,  NAVCENT,  and  ONI).  The  workflow  model  illustrates 
the  processes  required  to  execute  two  scenarios:  (1)  tracking  a  vessel  of  interest  (VOI)  and  (2) 
handing  off  the  VOI  from  one  numbered  fleet  commander  to  another.  These  scenarios  implement 
the  first  two  scenarios  envisioned  in  the  draft  MDA  Spiral  1  Scoping  Document1,  and  they 
arguably  incorporate  several  of  the  remaining  Spiral  1  scenarios.  We  will  expand  the  workflow 
models  to  address  additional  scenarios  as  needed. 


1  The  draft  Scoping  Document  for  Navy  Maritime  Domain  Awareness  (MDA)  Spiral  1  Prototype,  version 
2,  1 1  OCT  2007  specifies  the  following  scenarios:  (1)  Track  vessels  via  fusion  of  multiple  data  sources 
across  the  maritime  knowledge  base,  incorporating  both  current  and  archived  data;  (2)  Vessel  Tracking 
handoff  across  Fleet  Areas  of  Responsibility  (AORs);  (3)  Collaborate  between  nodes;  (4)  Establish  a  near- 
real  time  maritime  Common  Operational  Picture  that  provides  access  to  tactical,  operational,  and  strategic 
data  elements;  (5)  Improve  timely  transfer  of  vessel  data  collected  at  the  tactical  level  to  appropriate 
operational  and  strategic  nodes  (e.g.,  Expanded  Maritime  Interception  Operations  (EMIO),  AARs,  and 
track  updates);  (6)  Enable  participating  Navy  commanders  to  network  globally  through  real-time 
collaboration  and  information  sharing,  supporting  decision  making  at  the  strategic,  operational,  and  tactical 
levels  across  the  range  of  military  operations  (ROMO). 
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Figure  2.  MDA  Workflow  for  Observe,  Orient,  Decide  and  Act. 


7 


CIFC  Tactical  Forces  NAVCENT  MOC 

CTF5X,  15X 


NAVCENT  MDA  “As-Is”  Boarding  Process 


Regional  analyst 
continue  collecting, 
fusing,  and 
disseminating 
info  on  Vol 
(3,4,8)  [la,lb,2a^b] 


Request  to 
conduct 
VBSSonVoI 
fromTactical 
Commander 
(6) 


CNFC  C  hat/Email 


Tactical  Assets  provide 
ISR  support  and 
corresponding  Vol 
query  data  back  to 
MOC  with  request  to 
conduce  VBSS 
(2,6) 


CNFC /SIPR 
Chat /Email 


MOC  Intel  analyst 
will  continue 
collecting, 
fusing,  and 
disseminating 
info  on  Vol 
(3,4,8)  [la,lb,2a^b] 


Cont  nue 


Surw 


Co  itinue 
Surveillance 


unclasp 


Boarding  intelligence  available 
on  SEAPORT  for  Tre-boarding’ 
Analysis  or  Tost-boarding’  Fusion 
(l,3,5,6y9)[4a,4b] 


CMFC/  SIPR 
xmission  of  data 
^to  SEAPORT 


illarre 


Biometrics  Data 
sent  to  BFC 
inWV  for 
Anafysis 

PA5 ,6,9)  [4a,4b] 


UNCLAS  / 
>IPR  xmission 
:>fdatato  BFC 


0 


CIFC  is  boarding 
authoritative  source 
m  NAVCENT 

(.6) 


Task  assets  to 
continue 
tracking  and 
Surveillance  on  Vol 


CIFC  checks  Vol 
query  data  against 
organic  database, 
fishing  database, 
and  regional  SMEs 
like  MARLO 
(5,6)  [4a,4b] 


Board  ^ 
or  continue 
to 

surveill 


CTFwill  task  US  N 
or  Coalition  assets 
to  Board  Vol 
(2,6) 


5B)  US  ship  executes  VBS: 

(1,3, 5,6,9)  [4a,4b] 

Coalition  ship  executes  VBS^ 
(3,5,6, 9)  [4a,4b] 


Conduct  VBSS 


CNFC/ 

SIPR 


CIFC(?)  Issues 
Frago  Order 


All  Biometric  hits 
forwarded  to  ONI  /Positive 
matches  anafyaed  by  ONI  Biometric 
Anafysis  Cell  (BAC)  /BAC  produces 
BIAR  and  forwards  to  MOC 
for  reoommendation 


UNCLAS 


NCIS  MTAC 
AndFO 
Analy^ 
Boarding  Data 
p,2,3,5,6,9)  [4a,4b] 


SIPR 


MOC  receives 

analysis  of 
Boarding  Data 
and  provides 
disposition 
Instructions 
(2) 


CNFC  or  SIPR 


riJ] 


Email 


Tactical  Commander 
completes  boarding 
by  following 
disposition 
Instructions 


I  1  Indicates  a  function  or  task  that  is  sometimes  also  conducted  by  the  ONI 

1  1  Indicates  a  function  or  task  that  has  no  arrow  going  out 

Indicates  a  function  or  task  that  has  no  arrow  going  in 
I  I  Indicates  a  function  or  task  that  has  no  arrow  going  in  or  out 
Indicates  a  function  or  task  that  needs  more  fleshing  out 


Page  5  of  9 

MDA  Workflow  v12 


[SPIRAL  1  Capabilities] 

(SPIRAL  1  Technologies) 

1  a.  Enhanced  Vessel  Tracking 

1.  EMIO-CENTRIX 

lb.  Anomaly  Detection  (Vessels) 

2 .  Vessel  T racking  -  CMA 

2a.  Threat  Detection  (Vessels) 

3.  Anomaly  Detection  &  Pattern  Recognition 

2b .  Threat  Detection  (People) 

-  CMA,  TAANDEM,  FASTC2AP 

3.  E-MIO  (People) 

4 .  Threat  Detection  -  FASTC2AP 

4a.  N on-classified  Collaboration  Dissemination 

5.  Geospatial  Collaboration  -  Google  Apps 

4b.  Geospatial  Collaboration 

6.  Google  Fusion  Servers 
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6 .  Data  Sharing  with  Law  Enforcement 

8.  TRIPWIRE  -  ONI 

7 .  Data  Sharing  with  Coalition  Allies 

9.  LiNX-NCIS 

Figure  3.  NAVCENT  MDA  Process  Workflow  for  Current  (fAs  Is’)  Boarding  Process. 
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[SPIRAL  1  Capabilities] 

1  a.  Enhanced  Vessel  Tracking 

lb.  Anomaly  Detection  (Vessels) 

2a.  Threat  Detection  (Vessels) 

2b .  Threat  Detection  (People) 

3.  E-MIO  (People) 

4a.  N on-classified  Collaboration  Dissemination 

4b.  Geospatial  Collaboration 

5 .  Port  &  Coastal  Surveillance 

6 .  Data  Sharing  with  Law  Enforcement 

7.  Data  Sharing  with  Coalition  Allies 


(SPIRAL  1  Technologies) 

1.  EMIO-CENTRIX 

2 .  Vessel  T racking  -  CMA 

3.  Anomaly  Detection  &  Pattern  Recognition 
-  CMA,  TAANDEM,  FASTC2AP 

4 .  Threat  Detection  -  FASTC2AP 

5.  Geospatial  Collaboration  -  Google  Apps 

6.  Google  Fusion  Servers 

7.  Magnet  -  USCG 

8.  TRIPWIRE -ONI 

9.  LiNX-NCIS 


Figure  4.  MDA  ‘As  Is’  Track  and  Surveillance  w /  Handoff  from  MOC  to  MOC. 
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A  second  round  of  interviews  was  conducted  to  refine  these  workflow  models  with  site  visits  to 
ONI,  NAVCENT,  and  MIFC.  Specifically,  the  interview  protocol  for  these  site  visits  was 
designed  to:  (1)  identify  additional  tasks  and  task-task  relationships,  define  them  operationally, 
and  identify  potential  failure  points;  (2)  identify  infonnation  input  requirements,  (3)  identify 
information  outputs,  (4)  specify  requirements  regarding  human  resources,  technology,  and  time; 
and  (5)  specify  constraints  regarding  human  resources,  technology,  and  time.  These  attributes  on 
MDA  tasks  are  represented  in  scenarios  as  needed. 

Process  Engineering  Workshop 

A  process  engineering  workshop  (PEW)  was  held  at  the  Naval  Postgraduate  School  (NPS), 
Monterey,  CA,  15-17  January  2008  to  accomplish  several  critical  steps.  The  objectives  of  the 
PEW  were  to: 

•  Refine,  extend,  and  validate  a  process  model  of  Maritime  Domain  Awareness 

•  Define  attributes  of  the  activities  that  constitute  MDA,  specifically  information 
requirements,  processing  activities,  products,  and  resource  (time,  manning)  requirements 
(reported  in  Freeman,  Heacox,  and  MacKinnon,  2008) 

•  Specify  which  MDA  activities  may  benefit  from  Spiral  1  technologies 

•  Develop  concepts  for  assessing  the  effects  of  technology 

•  Identify  barriers  to  fielding  MDA  Spiral  1  technologies 

Representatives  of  the  following  organizations  participated  in  the  PEW:  Assistant  Secretary  of 
the  Navy  for  Research,  Development,  and  Acquisition  (ASN  RDA),  Commander,  THIRD  Fleet 
(C3F),  Commander,  Operational  Test  and  Evaluation  Force  (COTF),  Office  of  the  of  the  Under 
Secretary  of  the  Navy,  Defense  Information  Systems  Agency  (DISA),  HFE  LLC,  Joint 
Interoperability  Test  Command  (JITC),  METRON,  Maritime  Intelligence  Fusion  Center, 
Atlantic  (MIFCLANT),  Maritime  Intelligence  Fusion  Center,  Pacific  (MIFCPAC),  U.S.  Naval 
Force,  Central  Command  (NAVCENT),  Naval  Network  Warfare  Command  (NNWC),  Naval 
Criminal  Investigation  Service  (NCIS),  United  States  Northern  Command  (NORTHCOM),  NPS, 
Naval  Research  Laboratory  (NRL),  Naval  Warfare  Development  Center  (NWDC),  Office  of 
Naval  Intelligence  (ONI),  staff  of  the  Chief  of  Naval  Operations  (OPNAV),  PMW  120,  and 
Space  and  Naval  Warfare  Systems  Center  (SPAWAR).  Also  participating  were  subject  matter 
experts  (SMEs)  from  several  of  the  MDA  Spiral  1  technologies,  domain  experts  (‘gray  beards’), 
representatives  from  the  Trident  Warrior  2008  (TW08)  operational  experiment  where  many  of 
the  MDA  Spiral  1  technologies  will  be  assessed,  and  members  of  the  assessment  team  (NPS, 
Aptima,  Pacific  Science  &  Engineering,  WBB  Inc.). 

Workflow  review.  The  PEW  participants  reviewed  several  MDA  OV-6c  workflow  diagrams 
developed  by  NAVNETWARCOM  from  NPS  workflow  data.  These  diagrams  were: 
‘NAVCENT  MDA  Process’  (as  well  as  a  summary  diagram  for  this  workflow  (version  11)), 
‘Provide  MDA  Info-NMIC’,  and  ‘RFI  Processing-NMIC’.  Participants  recommended  revisions 
to  the  activities,  activity-activity  precedence  (links),  and  clustering  of  activities.  The  number  of 
revisions  was  modest,  and  participants  indicated  that  these  workflows  are  generally  correct.  PEW 
participants  recommended  revisions  to  the  NAVCENT  MDA  Process  workflow  to  generalize  it 
so  that  it  potentially  serves  MOCs  and  organizations  other  than  NAVCENT. 

Goals  for  the  PEW  were  to:  (1)  validate  the  initial  workflow  models  of  the  process  with  subject 
matter  experts  (SMEs)  from  the  representative  commands,  (2)  discuss  how  the  new  tools 
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developed  for  MDA  will  change  the  process,  and  (3)  develop  new  process  workflow  models  to 
depict  the  envisioned  processes  once  the  tools  are  incorporated  into  the  MDA  process.  Questions 
that  were  posed  to  the  subject  matter  expert  (SME)  attendees  are  listed  in  Table  2. 


Table  2.  Objective  Translation  of  User-Defined  Measures. 


Number  of 

Tracks 

1.  What  is  the  range  of  the  number  of  vessels  the  watch  team  must  track  at  any  given  moment'- 

2.  How  often  must  track  data  be  updated? 

3.  If  required,  how  often  is  manual  track  updating  needed? 

4.  What  sorts  of  vessel  data  (cargo,  destination,  crew,  hull  type,  flag)  are  required  as  part 
of  track  data? 

5.  How  many  AIS  tracks  are  typically  received? 

6.  How  many  AIS  tracks  are  typically  displayed? 

7.  Does  the  source  of  track  data  need  to  be  easily  discernable? 

8.  Do  sources  of  track  data  need  to  be  filtered? 

9.  How  many  tracks  from  radar  can  be  displayed? 

10.  How  many  tracks  from  radar  need  to  be  displayed? 

1 1 .  How  many  tracks  from  ELINT  can  be  displayed? 

12.  How  many  tracks  from  ELINT  need  to  be  displayed? 

Data  Archival 

13.  What  is  the  current  capability  to  archive  vessel  data? 

14.  How  much  more  storage,  if  any,  is  required  to  archive  vessel  data? 

15.  What  is  the  maximum  number  of  collating  (fusing)  events  a  watch  team  can  coordinate? 

16.  What  is  the  maximum  number  of  collating  (fusing)  events  that  ONA  could  coordinate? 

17.  How  many  tracks  would  have  been  identified  earlier  if  harbor  or  port  information 
(underway  times,  arrival  times,  cargo,  crew)  were  readily  available? 

18.  How  many  “heads-up”  vessel  track  (cargo,  destination,  crew,  hull  type,  and  flag)  events 
from  a  nearby  MOC,  have  taken  place  over  a  given  month? 

Information 

Reachback 

19.  How  many  requests  for  vessel  data  have  been  made  which  require  global  accessibility? 

20.  How  many  ONI  (Office  of  Naval  Intelligence)  requests  for  information  are  typically 
made  in  a  given  day? 

Overarching 

Technical 

21.  How  manv  conflicts  have  occurred  in  a  given  week,  over  releasabilitv  of  the  present  MDA 
technology  (GCCS-M,  Centrix,  SIPR,  chat,  JWICS,  etc.) 

22.  How  manv  conflicts  have  occurred  in  a  given  week,  over  usability  of  the  present  MDA 
technology  (GCCS-M,  Centrix,  SIPR,  chat,  JWICS,  etc.) 

23.  How  manv  conflicts  have  occurred  in  a  given  week,  over  reliability  of  the  present  MDA 
technology  (GCCS-M,  Centrix,  SIPR,  chat,  JWICS,  etc.)? 

24.  How  manv  conflicts  have  occurred  in  a  given  week,  over  automation  of  the  present  MDA 
technology  (GCCS-M,  Centrix,  SIPR,  chat,  JWICS,  etc.)? 

25.  How  manv  conflicts  have  occurred  in  a  given  week,  over  supportabilitv  of  the  present 
MDA  technology  (GCCS-M,  Centrix,  SIPR,  chat,  JWICS,  etc.) 

26.  How  manv  conflicts  have  occurred  in  a  given  week,  over  maintainability  of  the  present 
MDA  technology  (GCCS-M,  Centrix,  SIPR,  chat,  JWICS,  etc.) 
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Scenario  for  Spiral  1  Assessment  of  Technologies 

The  MDA  Spiral  1  effort  focuses  on  a  single  scenario  with  the  following  six  identifiable  goals: 
(a)  track  vessels  via  fusion  of  multiple  data  sources  across  the  maritime  knowledge  base, 
incorporating  both  current  and  archived  data;  (b)  vessel  tracking  handoff  across  Fleet  areas  of 
responsibility  (AORs);  (c)  collaborate  between  nodes;  (d)  establish  a  near  real-time  common 
operation  picture  (COP);  (e)  improve  transfer  of  vessel  data  collected  at  the  tactical  level 
(extended  maritime  interdiction  operations  (EMIO));  and  (f)  enable  Navy  commanders  to 
network  globally  through  real-time  collaboration  and  information  sharing,  supporting  decision 
making  at  the  operational  and  tactical  levels  across  the  range  of  military  operations  (ROMO). 
(PEO  C4I)  The  scenario  involves  information  sharing  across  multiple  areas  of  responsibility 
(AORs)  of  a  suspect  vessel  in  transit  from  the  Middle  East  to  the  vicinity  of  Hawaii.  Ultimately, 
the  information  sharing  and  collaboration  would  result  in  an  order  to  board  a  tactical  unit,  such 
that  each  of  the  goals  stated  above  would  be  tested. 

Using  the  process  described  above,  the  NPS  team  documented  the  workflow  of  Maritime 
Domain  Awareness  for  a  select  set  of  organizations:  Afloat  Units;  Biometrics  Fusion  Center 
(BFC);  Boarding  Party;  MIFC;  COMPACFLT;  Combatant  Commanders  (COCOM);  Maritime 
Liaison  Officer  (MARLO);  MIFCLANT;  MIFCPAC;  NAVCENT  MOC;  NCIS;  Naval  Maritime 
Intelligence  Center  (NMIC);  and  Subordinate  Commanders  and  Staff.  These  organizations  were 
chosen  in  part  because  the  scenario  used  in  interviews  involved  a  suspect  vessel  en  route  from 
the  Middle  East  towards  the  West  Coast  of  the  U.S.  Other  scenarios  might,  in  the  future,  engage 
other  organizations  and  invoke  other  activities.  The  workflow  represents  the  activities  of  the 
NAVCENT  MOC  (109  activities)  and  Naval  Maritime  Intelligence  Center  (NMIC)  (50 
activities)  in  the  greatest  detail.  The  remaining  organizations  are  represented  by  a  dozen  tasks  or 
fewer. 


TOOLS  TO  SUPPORT  MDA 

A  suite  of  tools  have  been  developed  for  the  Spiral  1  phase  and  the  prototypes  will  be  assessed 
during  the  Trident  Warrior  08  experiment  where  various  venues  exist  to  assess  the  level  of 
improvement,  as  well  as  other  venues.  The  Spiral  1  effort  focuses  on  establishing  a  core  net- 
centric  set  of  capabilities  that,  at  multiple  levels  of  security,  will  feed  and  fuse  many  data  streams 
into  a  coherent  common  operational  picture  while  providing  improved  analytical  and 
collaboration  tools.  The  following  section  describes  the  tools  and  the  capabilities  they  provide 
for  the  Spiral  1  phase  of  MDA. 

Comprehensive  Maritime  Awareness  (CMA).  Comprehensive  Maritime  Awareness  (CMA) 
was  developed  to  support  anomaly  detection  by  collecting  track  information  and  identifying 
behavior  that  deviates  from  normal  patterns  of  behavior,  e.g.,  relationships  between  vessels  and 
cargo.  CMA  supports  sharing  information  across  all  CMA  nodes  from  NMIC,  Coast  Guard, 
Combatant  Commanders,  Coalition  partners,  and  tactical  level  commands  and  supports  a  number 
of  tasks  and  functions  that  are  part  of  the  overall  MDA  process.  CMA  supports  generation  of 
tipper  information  (the  initial  step  (first  box)  in  the  workflow  diagram)  via  its  object  tracking 
capability:  objects  include  people,  cargo,  financial  transactions,  vessels,  vessel  ownership  and 
flag,  etc.  For  example,  connecting  the  name  of  a  crew  member,  to  related  persons,  to  financial 
transactions,  to  a  terrorist  group  can  be  accomplished  in  a  much  shorter  time  to  provide  tipper 
information  on  a  vessel  of  interest  (VOI),  person  of  interest  (POI),  cargo  of  interest  (COI),  and 
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so  on.  CMA  extracts  infonnation  from  a  wide  range  of  sources  and  integrates  this  multi-source 
data.  CMA  is  expected  to  improve  the  process  by  automatically  correlating  across  many 
information  sources  to  reduce  the  time  required  to  perform  critical  tasks  (e.g.,  reduce  the  time 
required  to  accomplish  a  task  that  currently  takes  many  hours,  down  to  minutes). 

CMA  is  expected  to  perform  a  more  comprehensive  correlation  across  multiple  sources  of 
information  (-350  databases)  due  to  automation  of  this  process.  For  example,  the  correlation 
process  is  accomplished  in  near-real  time  as  opposed  to  taking  many  hours.  An  explanation  is 
provided  to  the  user  along  with  any  tipper  information,  including  all  sources  used  by  the  system 
that  have  information  that  contributed.  CMA  will  be  installed  and  used  at  all  levels  depicted  in 
the  workflow  diagram,  i.e.,  from  ONI,  NAVCENT,  CFIC,  to  tactical  level  forces,  and  will  be 
used  in  slightly  different  ways  depending  on  the  organizational  level  depicted  in  the  workflow 
diagram. 

It  is  anticipated  that  the  ability  to  automate  the  search  and  correlation  process  across  multiple 
sources  represents  a  paradigm  shift  in  that  a  significant  reduction  is  expected  in  the  amount  of  an 
analyst’s  time  that  will  be  required  for  this  part  of  the  process.  By  automating  this  portion  of  the 
overall  task  it  should  free  up  the  analysts  to  do  the  higher  level  analysis  that  is  critical.  An 
additional  feature  incorporated  in  CMA  is  persistent  tracking  of  200  vessels.  This  history  of  a 
track  can  be  pulled  into  the  analysis,  e.g.,  ports  visited  along  the  route  of  the  vessel.  CMA  is 
based  on  the  premise  that  greater  transparency  of  existing  data  will  translate  into  greater  security. 

Track  Assessment  and  Anomaly  Detection  -  Maritime  (TAANDEM).  TAANDEM  is  a  track 
processing  system  to  identify  motion  that  is  inconsistent  with  previous  behavior  patterns.  This 
feature  is  based  on  processing  archived  data  to  determine  key  parameters  of  motion,  and  to 
design  key  detection  statistics.  The  prediction  and  activity  monitoring  component  provides  real¬ 
time  track  processing,  the  ability  to  identify  deviant  tracks,  and  facilitates  the  preparation  of  drill 
down  evidence  data  for  display.  TAANDEM  provides  input  to  CMA  in  the  form  of  annotated 
tracks. 

Alerts  are  generated  by  a  specific  factor  that  violates  a  boundary,  which  can  involve  anything 
that  can  be  computed  on  a  track.  TAANDEM  generates  the  degree  of  deviance  from  normalcy 
for  any  of  these  factors.  The  current  plan  is  to  let  the  user  reconfigure  settings  (such  as, 
parameters  for  boundaries,  degree  of  deviance  acceptable). 

MAGNET.  MAGNET  uses  agent  logic  to  obtain  data,  using  a  web  browser  to  query  other  data 
sources.  MAGNET  is  installed  at  the  Coast  Guard,  MIFCPAC.  The  web  browser  is  used  to  query 
the  system,  regarding  such  topics  as  weather,  ships  in  a  specific  area,  and  so  on. 

Fast  Connectivity  for  Coalition  Agents  Program  (FastC2AP).  FASTC2AP  uses  agent  tech¬ 
nologies  to  create  “watch  stander  composeable  automation,”  where  watch  standers  use  agents  to 
define  and  automate  information  management  tasks.  FastC2AP  is  based  on  a  Defense  Advanced 
Research  Projects  Agency  (DARPA)  developed  predecessor  tool,  and  uses  PACFLT  databases. 
The  FASTC2AP  tool  set  is  web-enabled  and  uses  data  from  incoming  e-mails,  Chat,  Global 
Command  and  Control  System-Maritime  (GCCS-M),  and  various  databases  from  around  the 
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world.  Outputs  include  on-screen  alerts,  e-mails,  and  over-the-horizon  (OTH)  Gold  messages 
that  update  GCCS-M  tracks. 

FastC2AP  is  a  collection  of  software  tools,  implemented  through  a  web  portal,  which  supports 
operators  as  they  manage  situational  awareness  and  track  behavior  within  their  AOR.  FastC2AP 
supports  watch  standers  and  analysts  by  automating  many  of  the  maritime  track  data  manage¬ 
ment  tasks  they  typically  perform  now.  The  FastC2AP  portal  allows  watch  standers  to  (a) 
process  vast  amounts  of  a  data,  (b)  access  multiple  databases,  and  (c)  return  relevant  data  for 
processing  by  analysis,  fusion,  and  exploitation  agents. 

E-MIO  Wireless.  Electronic  Maritime  Interdiction  Operations  (E-MIO)  Wireless  provides 
satellite  transmission  and  reception  for  non-biometric  boarding  data  (manifests,  etc.)  and 
automatically  ingests  data  into  authoritative  databases. 

Google  Applications  and  Chat.  Google  Applications  and  Chat  provide  a  collaboration  tool  that 
includes  alerts,  chat,  blogs,  calendar,  tabbed  web  portal,  and  productivity  tools.  Data  are  stored  in 
a  secure  Navy  enterprise  maintained  by  Google;  this  tool  supports  sharing  between  the  Navy, 
Department  of  State,  Department  of  Justice,  etc. 

Global  Trader.  Global  Trader  supports  queries  about  cargo  data  and  several  types  of  automated 
analyses:  anomaly  detection  (including  both  statistical  and  machine  learning),  pattern  matching, 
and  clustering,  and  provides  alerts  with  supporting  evidence. 

TRIPWIRE.  TRIPWIRE  mines  unstructured  text  data  and  alerts  analysts  to  messages  of 
interest,  based  on  user-defined,  persistent  alerting  rules.  This  tool  can  perform  more  than  1.5 
million  transactions  in  a  growing  database. 

Many  questions  remain  regarding  the  effectiveness  of  the  prototype  MDA  tools.  These  questions 
include  where  to  set  detection  thresholds  for  anomaly  detection,  i.e.,  how  many  degrees  of 
difference  are  appropriate?  Other  issues  also  pertain  to  the  tradeoff  between  false  alarms  rate  and 
missing  a  critical  event.  Reliance  on  databases  raises  the  issue  of  the  reliability  of  databases  and 
their  associated  update  rates. 

Technical  Capabilities:  Description  of  Technology  Features 

From  the  MDA  Scoping  Document,  the  Spiral  1  effort  focuses  on  establishing  a  core  set  of  net- 
centric  capabilities  that,  at  multiple  levels  of  security,  will  feed  and  fuse  many  data  streams  into  a 
coherent  common  operational  picture  while  providing  improved  analytical  and  collaboration 
tools.  Table  3  presents  a  brief  summary  of  the  MDA  tools,  features  provided,  and  their  associated 
benefits. 

Table  3.  Spiral  1  Tools,  Features  Provided  and  Benefits  of  Tools. 


MDA  Tool 

Features  Provided 

Benefits  of  Tool 

CMA 

Anomaly  detection 

•  Automates  a  previously  manual  process 

Information  sharing  across  organizational  nodes 

•  Facilitates  collaboration 

Object  tracking 

•  More  tracks  can  be  monitored 

Integrates  multi-source  data,  includes  search 
agents,  confidence  reporting,  and  remote  access 

•  More  comprehensive  correlation 
across  multiple  sources  of  information 
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Automatic  data  correlation 

•  Reduced  time  to  perform  tasks 

FASTC2AP 

Defines/  automates  information  management 
tasks 

•  Generates  alerts,  emails,  OTH  Gold 
messages  that  update  GCCS-M  tracks 

Helps  manage  situational  awareness 

•  Access  to  multiple  databases 

Processes  large  amounts  of  data  by  analysis, 
fusion,  and  exploitation  agents 

•  Helps  manage  situation  awareness  and 
track  behavior  in  an  area  of  responsibility 

TRIPWIRE 

Mines  unstructured  text  data  and  alerts  analysts 

•  Increased  relationships  analyzed 

User-defined  persistent  alerting  rules 

•  Reduced  latency 

MAGNET 

Intelligent  agent  logic 

•  Faster  access  to  data 

Web  browser  to  query  other  data  sources 

•  More  sources  queried  in  less  time 

System  queries  regarding  topics  of  interest  in  area 

•  Increased  number  of  analytic  models 

•  Increased  archived  data  for  analysis 

TAANDEM 

Real-time  track  processing 

•  Prediction  and  activity  monitoring 

Identify  deviant  tracks 

•  Real-time  track  processing 

Provides  annotated  tracks  with  alerts 

•  Rule-based  anomaly  detection 

E-MIO  Wireless 

Provides  satellite  transmission/  reception  for 
unclassified  boarding  data 

•  Automatically  inserts  data  into 
authoritative  databases 

Google  Applica¬ 
tions  and  Chat 

Collaboration  tool:  Alerts,  chat,  blogs,  calendar, 
tabbed  web  portal,  and  productivity  tools 

•  Supports  sharing  between  the  Navy,  Dept, 
of  State,  Dept,  of  Justice,  etc. 

Global  Trader 

Anomaly  detection,  pattern  matching/  clustering, 
and  provides  alerts  with  supporting  evidence 

•  Supports  queries  regarding  cargo  and 
data  analyses 

MARITIME  DOMAIN  AWARENESS  SPIRAL  1  ASSESSMENT  PLAN 

The  NPS  assessment  plan  provides  assessment  guidance  for  the  FY08  Spiral  1  events  of  the 
MDA  Science  and  Technology  (S&T)  Prototype  Acceleration  effort. 

Spiral  1  Assessment  Strategy 

The  MDA  2008  Spiral  1  assessment  efforts  focus  on  identifying  baseline  performance  para¬ 
meters  associated  with  the  broadly  stated  capabilities  for  the  present  state  of  MDA  technologies. 
Our  approach  was  to  develop  an  overarching  assessment  plan  to  refine  objectives,  develop 
measures  and  design  data  collection  instruments  so  that  analysis  of  the  tools  can  be  conducted 
and  the  results  and  recommendations  can  be  forwarded  to  decision  makers  for  continuing 
improvement  in  MDA.  The  NPS  assessment  effort  includes  developing  performance  metrics 
related  to  availability,  reliability,  timeliness,  usability,  and  accuracy.  Data  on  these  attributes  will 
be  collected  along  with  other  specific  parametric  data  that  will  be  used  to  define  the  existing 
MDA  performance  capabilities.  The  assessment  strategy  was  also  developed  to  assist  in 
establishing  the  operational  baseline  against  which  future  improvements  can  be  measured.  This 
data  will  be  collected  at  various  MDA  or  MDA-related  events,  and  other  venues  of  opportunity 
listed  in  the  schedule  for  Spiral  1 . 

The  NPS  assessment  plan  combines  input  from  many  sources  to  develop  an  assessment  plan  built 
on  previous  studies  that  will  enable  decision  makers  to  determine  MDA  effectiveness  and 
shortcomings.  We  began  by  developing  an  understanding  of  the  ‘present  state’  of  the  Spiral- 1 
technologies  as  documented  in  the  MDA  Scoping  Document.  This  was  followed  by  the  scenario 
development  that  is  presented  in  the  scoping  document.  The  scenarios  were  then  used  to 
formulate  anticipated  workflow  models  that  were  refined  from  both  theoretic  scenarios  and 
multiple  real-world  site  visits.  The  goal  is  to  provide  decision  makers  a  concise,  repeatable,  and 
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highly-refined  methodology  to  improve  MDA  capabilities  and  to  provide  input  that  will  support 
future  budgetary  allocations  based  on  this  analytical  framework. 

The  Spiral  1  assessment  goals  include  the  ability  to:  (a)  understand  the  capabilities  and 
limitations  of  the  present  state  MDA  prototype  functional  areas;  (b)  establish  a  baseline  of 
operational  parameters  traceable  to  the  MDA  test  and  evaluation  plan;  (c)  execute  a  Risk 
Reduction  Limited  Objective  Demonstration  prior  to  the  initial  operational  capability  (IOC);  and 
(d)  identify  areas  for  improvement  that  may  be  employed  in  follow-on  Spiral  efforts. 

Use  case  development  and  use  in  planning.  The  purpose  for  the  use  case  analysis  is  to  create  a 
set  of  representative  use  cases  (threads)  that  examine  MDA  as  a  system  to  include  people, 
elements  of  doctrine,  concept  of  operations  (CONOPs),  information  exchange  needs,  methods 
for  transporting  infonnation  requirements,  and  tasks  that  will  stimulate  the  tasks  and  various 
exchanges  to  reveal  Spiral- 1  performance  capabilities  and  future  needs  requirements.  An  OV-5 
workflow  model  (IDEF  view  in  UML)  and  an  OV-6c  event  trace  model  are  the  primary 
mechanisms  by  which  to  create  the  use  cases,  with  a  data-flow  model  (OV-7)  providing  the 
information  exchange  requirements  that  meet  the  needs  of  watch  standers  and  decision  makers. 

The  range  of  tasks  at  the  user  level,  within  MHQ/MOC  is  being  defined  in  a  data  pull  from 
NAVCENT,  Commander,  Pacific  Forces  (CPF),  ONI,  and  other  organizations  at  this  time. 
Results  of  the  data  analysis  provide  an  input  to  the  development  of  the  DoDAF  views,  and  will 
be  used  by  the  Architecture  Working  Group  which  will  in  turn  produce  the  models  described 
above.  A  necessary  component  to  be  developed  is  the  scenarios  or  vignettes  that  encompass 
those  tasks  and  activities  needed  in  IOC  demonstrations. 

Metrics  and  Measures.  MDA  objectives  have  been  organized  into  a  three-level  structure  of 
objectives  and  metrics  that  facilitates  the  design  of  specific  experimentation  objectives.  This 
approach  has  been  used  and  refined  over  several  years  of  fleet  experimentation  (e.g.,  TRIDENT 
WARRIOR)  as  it  was  developed  for  Navy  Network-Centric  Warfare  Command 
(NAVNETWARCOM).  The  approach  we  are  using  entails  detailed  documentation  of  objectives, 
metrics,  and  all  aspects  of  the  measurement  process. 

MDA  Critical  Tasks 

•  Direct  Operational  Intelligence  Activities 

•  Process  and  Exploit  Collected  Operational  Information 

•  Produce  Operational  Intelligence  and  Prepare  Intelligence  Products 

•  Disseminate  and  Integrate  Operational  Intelligence 

•  Evaluate  Intelligence  Activities  in  the  JO  A 

•  Collect  and  Share  Operational  Information 

•  Acquire  and  Communicate  Operational  Level  Information  and  Status 

•  Assess  the  Operational  Situation 

•  Develop  a  Shared  Understanding  of  the  Situation 

•  Prepare  Plans  and  Orders 

•  Command  Subordinate  Operational  Forces 

•  Coordinate  and  Integrate  Joint/Multinational  and  Interagency  Cooperation 
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Metrics  are  needed  to  assess  the  effects  of  technology  insertion  relative  to  the  current  state  of 
capabilities  for  MDA.  Currently,  there  are  few  if  any  published  standards  that  define  the 
effectiveness  of  current  solutions  in  operations.  Moreover,  a  sufficiently  detailed  scenario  is 
needed  to  drive  testing  and  must  systematically  address  the  variety  of  MDA  data  types  (vessel, 
people,  cargo,  etc.),  reporting  products,  node  interactions,  and  time  course  of  activity  in  a 
problem  that  involves  discovery,  analysis,  and  prosecution  of  VOIs.  Particularly  important 
challenges  in  MDA  include  ISR  management,  collection  planning,  decisions  regarding  opposed 
and  unopposed  boardings,  and  tracking  neutrals.  In  addition,  scenario  designers  need  to  consider 
events  in  which  multiple  vessels  collaborate  in  a  threat  incident,  either  through  cargo  transfer 
between  vessels  or  by  synchronized  tactical  actions  of  two  or  more  vessels. 

Test  Objectives  Structure.  MDA  program  objectives  were  obtained,  however  these  do  not 
include  specific  objectives  for  the  various  tests.  Specific  test  objectives  will  be  developed  as  part 
of  the  NPS  assessment  planning  effort,  derived  from  MDA  program  objectives.  Test  objectives 
are  organized  into  the  following  six  test  areas:  (1)  supporting  technology  capabilities;  (2) 
information  processing  capabilities;  (3)  operations  performance  status;  (4)  workflow  status;  (5) 
guidance  quality;  and  (6)  E-MIO  operations  capabilities.  The  objective  categories  for  each  test 
area  are  shown  in  Table  4. 

Table  4.  Test  Areas  and  Objective  Categories  for  MDA  Program  Objectives. 


Test  Area 

Objective  Category 

Test  Area 

Objective  Category 

Supporting  Technology  Capabilities 

I 

nformation  Processes  Capabilities 

Monitoring  &  Collection 

Monitoring  &  Collection 

Processing/Fusion 

Processing/Fusion 

Analysis 

Analysis 

Dissemination 

Dissemination  > 

Archiving  &  Maintaining  Repository 

Archiving  &  Maintaining  Repository 

Operations  Performance  Status 

\ 

Workflow  Status 

Plan  Operations 

Structure 

Execution 

Organization 

Execution  Monitoring 

Group 

Execution  Assessment 

Individual 

Guidance  Quality 

E-MIO  Operations  Capabilities 

TTP  &  SOP 

Information  Acquisition 

CONOPS 

Mission  Planning 

Command  Guidance 

Boarding  Execution 

Standing  Orders 

Execution  Monitoring 

Information  Collection 

Information  Dissemination 

Information  Assessment 
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The  objective  categories  monitoring  and  collection,  processing/fusion,  etc.,  appear  in  two  test 
areas.  One  set  is  for  determining  the  capabilities  of  the  various  MDA  information  processes,  the 
other  is  for  determining  the  capabilities  of  the  systems  that  support  those  processes.  Specific 
objective  types,  their  associated  metrics,  and  objectives  have  been  defined  for  each  of  the 
objective  categories.  The  full  set  of  objectives  and  metrics  structure  is  currently  being  developed. 
E-MIO  is  a  special  test  area,  as  E-MIO  involves  a  specific  operation,  and  the  other  areas  are 
general  and  apply  to  the  full  range  of  MDA  activities.  Each  test  area  includes  several  objective 
categories  which  correspond  to  the  MDA  objectives. 

Metrics  Structure.  Metrics  are  based  on:  (1)  attributes;  (2)  measures  associated  with  each  goal 
and  attribute;  and  (3)  any  applicable  standards/thresholds  for  the  measures.  The  accepted  MOP, 
MOE,  MOU  structure  is  used,  with  Measures  of  Readiness  (MOR)  added  for  acquisition 
purposes.  MOR  applies  only  to  systems.  Measure  of  Effectiveness  (MOE)  refers  to  the  overall 
effectiveness  of  a  system,  people,  or  process  to  perform  its  stated  mission.  Measures  of  Perfor¬ 
mance  (MOP)  are  direct  measures  of  a  specific  performance  parameter  of  people,  process,  or 
system.  Measures  of  Utility  (MOU)  include  the  effectiveness  of  organizations,  humans, 
processes,  or  systems  for  supporting  operational  activities,  or  for  guidance  in  directing  those 
activities.  Measure  of  Readiness  (MOR)  is  used  to  determine  if  the  system  is  ready  for  fielding. 

Table  5  contains  the  MOE,  MOP,  MOU  attribute  structure.  For  example,  there  are  four  attributes 
for  measures  of  effectiveness  (i.e.,  accessible,  reliable,  capable,  and  usable)  and  under  each  one 
are  its  component  MOP  attributes.  There  are  also  four  attributes  for  MOU.  MOP-level  attributes 
for  MOU  are  not  shown;  use  of  the  MOP  is  appropriate,  although  most  MOU  determinations  are 
made  through  surveys.  Readiness  is  a  combination  of  effectiveness,  utility,  whether  there  is  a 
life-cycle  plan,  and  whether  trained  personnel  are  in  the  pipeline  for  use  of  the  system. 


Table  5.  MOE,  MOP,  MOU  Attribute  Structure  for  MDA  Assessment. 


Effective 

Accessible 

Reliable 

Capable 

Usable 

MOE 

Capacity 

Robust 

Sufficient 

Clear 

MOP 

Available 

Persistent 

Flexible 

Trusted 

ft 

Compatible 

Secure 

Accurate 

Manageable 

ft 

Extensive 

Assured 

Timely 

Relevant 

M 

Efficient 

Reach 

Compliant 

II 

Automatic 

Deployable 

II 

Military  Utility 


Improved 

Needed 

Applicable 

Wanted 


Effective 

Utility 

Life-Cycle 

Personnel 

MOU 


MOR 


Task  /  Attribute  /  Measure  Relationships.  Attributes  and  measures  have  meaning  only  when 
associated  with  an  activity  or  task.  Figure  5  depicts  the  various  types  of  task,  attribute,  and 
measure  associations  encountered  in  military  operations  assessments.  It  is  not  sufficient  to  test 
system  performance  in  isolation;  it  is  also  necessary  to  test  the  processes  that  systems  serve,  and 
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the  humans  and  organizations  that  execute  the  processes.  The  relationships  shown  in  Figure  5 
illustrate  the  types  of  assessments  that  are  needed. 


System 

Measures 


Information 

Acquisition 


System 

Measures  " 

Information 

Processing 


GUI/HSI  # _ 

Display 

Measures 

Systems 

Human 

Performance 
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Human  m 
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'  Teams 

@  Process 
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Individual  Info 
Components 


Information 
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N  etwork/Comms 

Info 

Measures  w 

Distribution 

Process 
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-•  Information 
Measures 


A 
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Attributes 


Process 

Attributes 
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Process 

Measures 


Decision-Process 

Attributes 


I  Operational  | 
|  Performance  | 


Human 
Attribute 

Human-Group  & 
Organization 


Figure  5.  Attribute  and  Measure  Types  and  Relationships  to  Systems  and  Activities. 

RECOMMENDATIONS 

We  make  approximately  twenty  recommendations  based  on  these  analyses.  (For  a  more  detailed 
reporting  on  this  effort  and  the  complete  list  of  recommendations  see,  Freeman,  Gallup, 
MacKinnon  &  Hutchins,  2008.)  These  recommendations  concern  training,  provisioning, 
evaluation,  management  of  technology  enabled  processes,  and  other  issues.  We  recommend  that 
the  Navy  build  upon  this  work  in  the  following  ways: 

•  Conduct  technology  assessments  that  focus  on  human  factors  issues:  trainability, 
usability,  utility,  and  the  fit  between  technology,  organizational  structure,  and  processes. 
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•  Extend  the  current  ‘as-is’  analysis  to  define  MDA  ‘to-be’  procedures  that  (1)  conform  to 
the  MHQ  w/  MOC  and  ONI  process  architectures  and  (2)  are  customized  to  the  needs  of 
specific  users. 

•  Develop  instruments  and  techniques  for  monitoring,  measuring,  and  managing  workflow 
at  critical  junctures  in  the  MDA  process. 

•  Implement  a  program  to  define  a  rapid,  tailorable  technology  training  program  and 
measure  its  effectiveness.  Adapt  both  the  training  and  technology  accordingly. 

Recommendations  from  Qualitative  Analysis 

(1)  Pay  particular  attention  to  planning  and  socializing  the  plan  to  train  and  maintain 
support  for  Spiral  1  technologies.  This  may  allay  concerns  in  COMPACFLT, 
NAVCENT,  and  elsewhere  that  the  technology  delivery  is  not  paying  sufficient 
attention  to  the  human  factor. 

(2)  Training  for  Spiral  1  technology  users  and  maintainers  should  be  rapid,  demonstrably 
effective,  and  customized  to  local  missions  and  procedures.  This  will  address 
concerns  that  technology  won’t  support  local  missions  and  processes  for  staffs  with 
high  turnover. 

(3)  Given  the  potential  that  staff  will  use  new  MDA  technologies  infrequently  in  some 
organizations,  they  may  forget  how  to  use  the  technologies  accurately  and  efficiently. 
Usability,  training,  and  technical  support  will  be  particularly  important  predictors  of 
success  in  these  organizations.  These  should  be  a  focus  of  assessment. 

(4)  Technology  providers  should  continue  to  brief  user  communities  concerning  the 
maturity  and  delivery  schedules  for  technologies.  This  may  help  users  to  prepare  for 
the  specific  capability  they  will  receive. 

(5)  The  accreditation  process  must  be  carefully  managed  across  the  Spiral  1  technologies. 
Lessons  learned  should  be  used  to  accelerate  that  process. 

Assessing  the  Effect  of  Spiral  1  Technologies 

PEW  participants  offered  ideas  for  assessing  the  effects  of  Spiral  1  technologies  on  tasks,  though 
there  were  few  specific  measures  suggested  and  no  perfonnance  standards.  These  strategies  for 
assessing  the  fit  of  technologies  to  tasks  fell  into  three  categories,  which  should  be  a  focus  of 
future  experiments  and  evaluations:  affects  on  (1)  access  to  infonnation  (that  was  previously 
inaccessible  by  the  performing  entity);  (2)  speed  of  decision  making;  and  (3)  accuracy  of 
decision  making. 

Future  Work 

NPS  will  proceed  with  its  planned  assessment  activities  and  the  majority  of  the  assessment 
events  will  occur  in  Spring/Summer  2008.  This  includes  collecting  and  refining  objectives,  as 
well  as  defining  each  objective’s  goals,  guiding  questions,  system  requirements,  measures  to  be 
recorded,  and  data  collection  plan.  The  assessment  plan  for  each  objective  will  be  placed  into  a 
retrievable  data  base:  FIRE  (ForceNet  Innovation  Research  Enterprise)  (Schacher,  2007). 
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